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1.   Introduction 

This  paper  attempts  to  summarize  the  effects  of  offering  compensa- 
tion to  respondents  on  consumer  expend' ture  surveys  based  on  the  results 
of  a  number  of  observational  and  experimental  studies  of  this  question 
undertaken  by  the  University  of  Illinois  and  other  institutions.  It  ad- 
dresses itself  to  the  evaluation  of  two  effects  on  such  surveys --coopera- 
tion and  report  validity.  Cooperation  we  define  here  as  equivalent  to 
the  response  rate,  that  is,  the  proportion  of  sample  households  contacted 
that  grant  the  desired  interview  (s) . 

Another  aspect  of  cooperation,  inducing  those  being  interviewed  to 
provide  more  complete  or  more  accurate  information,  is  subsumed  under  our 
second  major  heading,  report  validity.   It  is  defined  as  the  expenditures 
(in  units  or  amounts,  as  the  case  may  be)  reported  by  the  cooperating  house- 
hold in  relation  to  the  true,  unknown  expenditure.   In  the  absence  of  vali- 
dating information  in  most  of  these  studies,  it  is  not  possible  to  separate 
this  factor  into  its  two  components  of  more  complete  information  and  greater 
accuracy  of  information  that  is  reported.   Indeed,  all  that  can  be  assumed, 
as  will  be  shown  later,  is  that  under  the  circumstances  in  which  the  studies 
were  carried  out,  more  expenditures  means  greater  reporting  accuracy,  with 
the  partitioning  of  this  factor  between  completeness  and  accuracy  left 
to  later,  more  comprehensive  studies. 

We  begin  by  considering,  in  the  next  section,  the  rationale  for  of- 
fering compensation  in  consumer  expenditure  surveys,  some  of  the  types  of 
compensation  that  can  be  (and  have  been)  offered,  and  what  effects  a_ 
priori  reasoning  leads  us  to  expect  from  such  offers.  Though  any  such 
inferences  are  bound  to  be  colored  by  our  ex  post  knowledge  of  the  results 
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of  these  past  studies,  such  reasoning  should  nevertheless  serve  a  useful 
purpose  by  providing  a  more  rigorous  framework  for  evaluating  the  meaning 
and  significance  of  the  empirical  studies. 

Section  3  reviews  evidence  from  empirical  studies  of  the  effect  of 
compensation  on  cooperation,  while  Section  4  considers  what  information 
these  studies  provide  on  report  validity.  The  final  section  brings  together 
the  results  of  the  preceding  r.wo  sections  and,  with  these  results  as  a 
basis,  attempts  to  draw  some  generalizations  regarding  the  types  of  con- 
ditions under  which  compensation  would  seem  desirable  as  well  as  questions 
on  this  topic  that  would  constitute  the  basis  for  further  research. 

Before  beginning,  it  is  important  to  distinguish  among  different 
methods  of  collecting  expenditure  information,  since  compensation  procedures 
and  effects  can  vary  substantially  by  these  methods.   For  these  purposes, 
three  such  methods  can  be  distinguished.  One  method  is  to  collect  this 
information  by  a  one-time  interview  with  the  household,  as  was  done  in  the 
1960-61  and  1950  Consumer  Expenditure  Surveys  of  the  U.S.  Bureau  of  Labor 
Statistics.  The  interview  in  such  a  case  can  be  quite  long,  at  times  up 
to  eight  hours  (which  may  be  divided  into  two  or  three  sittings) ,  and  in- 
volves recall  of  expenditures  for  periods  varying  from  one  week  to  a  year 
depending  on  the  type  of  expenditure. 

A  second  method  entails  the  information  of  panels  of  households  who 
are  asked  to  report  expenditures  periodically  by  direct  interview,  either 
a  face-to-face  interview  or  by  telephone.  The  recall  is  much  shorter, 
usually  not  exceeding  three  months,  but  involves  much  more  cooperation  on 
the  part  of  the  household  over  time.  This  type  of  data  collection  method 
has  been  utilized  much  more  in  other  countries,  such  as  in  the  surveys  of 
income  and  expenditures  undertaken  by  the  Latin  American  group  of  research 
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institutes  coordinated  by  the  Brookings  Institution, (1)  and  in  various 
experimental  studies  to  be  reported  shortly. 

The  third  method  of  data  collection  also  involves  panels  of  house- 
holds but  with  expenditures  recorded  in  written  diaries  rather  than  reported 
orally.  These  diaries  usually  cover  short  periods  of  time,  usually  not 
more  than  a  week  or  two  but  sometimes  as  long  as  a  month.  The  use  of  these 
diaries  tends  to  provide  far  more  detail  on  expenditures  than  can  be  ob- 
tained otherwise  and,  at  least  theoretically,  enables  the  respondent  to  re- 
cord the  expenditures  immediately  after  they  have  taken  place  and,  hence, 
presumably  with  the  greatest  completeness  and  accuracy.   In  the  United 
States  these  diaries  have  been  used  primarily  for  supplementary  data  col- 
lection, mainly  for  food  and  related  expenditures,  though  they  are  being 
used  much  more  heavily  in  the  current  survey  of  consumer  expenditures  being 
conducted  by  the  U.S.  Bureau  of  Census  for  the  BLS. 

2.  Why  Compensate? 

The  general  attitude  in  the  past  has  been  that  compensation  was  not 
needed  on  expenditure  surveys  undertaken  by  governmental  agencies,  for 
two  reasons.  Since  the  data  were  being  sought  by  a  governmental  agency, 
it  was  felt  that  people  would  not  expect  to  be  paid  and  that  they  would 
consider  their  cooperation  as  a  public  service.   Second,  there  was  the 
further  feeling  that  when  such  data  were  sought  by  a  personal  interview, 
even  though  considerable  memory  effort  might  be  required,  the  respondent 
would  still  receive  psychic  rewards  from  the  social  interaction  with  the 
interviewer  in  which  he  (or  she)  is  the  dominant  figure.   It  seems  to  have 
been  essentially  for  these  reasons  that  for  many  years  in  the  United  States 
no  compensation  was  offered  on  expenditure  studies  by  government  agencies, 
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al though  marketing  research  firms  seeking  much  the  same  information  did 
offer  compensation,  and  are  still  doing  so. 

The  shift  toward  the  greater  use  of  diaries  and  written  records  on 
expenditure  surveys,  brought  about  by  the  large  body  of  evidence  on  the 
substantial  errors  that  exist  in  expenditure  data  obtained  by  recall,  has 
served  to  place  increasing  attention  on  possibly  offering  some  form  of 
compensation.   Rising  wage  levels,  increasing  labor  force  participation  by 
women,  and  a  tendency  to  involve  family  members  other  than  the  housewife 
in  these  interviews  have  all  served  to  accentuate  this  trend.   In  effect, 
the  rationale  is  that  record-keeping  and  other  forms  of  writing  are  in- 
herently distasteful  to  most  people,  and  they  have  to  be  encouraged  to  do 
so  by  means  of  special  incentives. 

While  marketing  research  firms  have  been  using  various  methods  of  com- 
pensation for  a  number  of  years,  their  use  in  expenditure  surveys  by  govern- 
mental agencies  has  only  recently  been  considered,  and  some  experiments 
evolving  from  this  consideration  are  discussed  in  the  later  sections  of 
this  paper.  Thus  it  is  not  surprising  that  most  survey  organizations  do 
not  compensate  respondents  for  one-time  interviews  while  diary-keepers 
are  usually  compensated  either  in  cash  or  with  gifts.   For  panels  where 
households  report  periodically  to  interviewers ,  there  is  some  tendency  to 
compensate  on  some  of  the  interviews,  especially  if  the  interview  is  long 
and  complex,  and  if  the  panel  will  be  used  for  more  than  a  year. 

One  other  issue,  for  which  we  have  only  limited  field  information, 
is  whether  the  need  for  compensation  is  related  to  the  type  of  survey  or- 
ganization. Thus,  some  researchers  argue  on  ethical  grounds  that  when 
expenditure  information  is  collected  for  commercial  purposes  by  a  profit- 
making  organization  the  respondent  should  be  compensated,  while  if  the 
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information  is  collected  by  a  governmental  or  other  non-profit  survey  or- 
ganization compensation  is  not  required. 

Compensation  can  take  many  forms.   The  most  obvious  form  is  a  cash 
payment  for  granting  the  interview.     s  has  been  used  by  some  commercial 
firms  but  has  been  used  very  little  by  governmental  agencies,  since  the 
amount  of  cash  that  can  be  offered  is  usually  not  considered  enough  to 
influence  the  respondent  to  cooperate  if  he  is  otherwise  inclined.  There 
are  some  exceptions,  however,  where  cash  seems  to  be  more  convenient.  Thus, 
where  record-keeping  is  not  required  but  long-term  cooperation  is  necessary, 
the  A.C.  Nielsen  Television  Panel  rewards  the  household  with  fifty  cents 
when  the  cartridge  is  removed  from  the  television  set  for  mailing. 

Merchandise  gifts  take  many  forms.  Most  pertinent,  perhaps,  is  the 
distinction  between  gifts  given  outright  in  exchange  for  a  single  interview 
and  gifts  given  over  a  period  of  time  in  exchange  for  cooperation  in  a 
panel.  The  MRCA  National  Consumer  Panel  rewards  households  with  gifts  of 
their  choice  from  an  attractive  catalog  similar  to  the  one  used  by  trading 
stamp  companies.  The  advantage  of  gifts  rather  than  cash  is  that  gifts 
may  sometimes  be  purchased  at  reduced  prices  by  the  researchers,  so  that 
the  value  to  the  respondent  is  greater  chan  the  outlay  to  the  survey  or- 
ganization.  In  addition,  gifts  act  as  a  continuing  reminder  each  time 
they  are  used  or  observed. 

A  combination  approach  may  also  be  used,  such  as  offering  one  gift 
to  a  household  on  the  first  interview  to  induce  them  to  begin  to  cooperate 
and  to  offer  something  else  toward  the  completion  of  their  service  on  the 
panel.  Needless  to  say,  numerous  alternatives  are  possible,  and  as  of 
this  writing  very  little  information  is  available  on  the  relative  effec- 
tiveness of  these  different  alternatives. 
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The  focus  of  this  paper  is  on  the  effects  of  compensation  on  cooper- 
ation and  report  validity  relative  to  expenditure  surveys  conducted  by  non- 
profit or  governmental  agencies .  To  the  extent  that  compensation  has  any 
effect  in  these  instances,  we  would  expect  that  merchandise  offerings  would 
have  more  effect  than  the  offer  of  an  equivalent  cash  amount,  that  merchan- 
dise of  a  less  commercial  nature  would  be  more  effective  than  other  types 
of  merchandise,  and  that  in  the  case  of  a  panel  study  several  gifts  spread 
over  time  would  be  more  effective  than  the  same  gift  offered  at  one  point 
in  time.  We  would  also  expect  for  economic  reasons  that  the  effects  would 
be  more  pronounced  on  lower  income  and  poorly  educated  families.  Although 
none  of  the  studies  to  be  reported  in  the  remainder  of  this  paper  were  de- 
signed specifically  with  these  individual  hypotheses  in  mind,  it  is  of 
interest  nevertheless  to  examine  the  extent  to  which  these  hypotheses  are 
supported  by  the  data. 

3.  Effect  on  Cooperation 

Seven  experiments  or  studies  of  the  effect  of  compensation  on  cooper- 
ation are  reviewed  in  this  section.  Only  one  of  these  relates  to  a  one-time 
interview.  Of  the  other  six,  two  were  panel  studies  seeking  data  by  per- 
sonal interview  and  four  were  panel  studies  using  the  diary  approach. 

a.  One-time  interviews 

There  is  little  evidence  on  the  effects  of  compensation  for  single 
face-to-face  interviews,  although  there  is  substantial  evidence  that  com- 
pensation improves  cooperation  on  mail  questionnaires.   (2,  pp.  94-100.) 
In  the  one  personal  interview  experiment  that  has  come  to  our  attention, 
on  obtaining  reports  of  savings  accounts  conducted  by  the  Survey  Research 
Center  in  1959,  the  cooperation  rate  was  67  percent  among  those  offered 
no  compensation  arid  was  much  less,  52  percent,  when  respondents  were 
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offered  ten  dollars. (6)  This  drop,  however,  may  have  been  due  to  suspicion 
on  the  part  of  respondents  of  the  purposes  of  the  study  and  also  to  mis- 
understandings about  the  method  by  interviewers,  as  noted  by  the  authors. (6, p. 127) 
Although  isolated  instances  have  been  encountered  of  people  refusing  to 
cooperate  with  a  university  study  if  offered  remuneration- -the  boomerang 
effect  ("What  business  has  a  university  doing  in  giving  people  money  when 
they're  always  asking  for  money. ..?") --it  is  hard  to  imagine  such  a  phenom- 
enon taking  place  on  a  wide  scale. 
b.  Panel  personal  interviews 

A  noncontrolled  test  of  the  effect  of  compensation  on  households  in- 
terviewed periodically  was  also  carried  out  as  part  of  the  Consumer  Savings 
Project  during  the  course  of  a  panel  study  of  saving  behavior  in  Chicago  in 
1957-58.(4)  At  the  interviewer's  discretion,  an  initial  gift  of  merchan- 
dise or  a  magazine  subscription  which  cost  the  project  between  $2.50  and 
$4.50  was  offered  to  the  respondent.   Circumstantial  evidence  based  on  in-   .' 
terviewer  evaluations  indicated  that: 

An  advance  offer  of  a  gift  helped  secure  cooperation  in 
about  one-fifth  of  the  background  interviews.  The  gift  was 
felt  to  be  particularly  effective  among  lower  income  groups, 
whereas  when  higher  income  people  were  offered  gifts,  the 
interviewer  was  often  rebuked  for  doing  so. 

Of  those  offered  gifts  about  nine  out  of  ten  accepted. 
Though  it  is  not  clear  whether  the  acceptance  of  a  gift  was 
cause  or  effect,  those  who  accepted  gifts  were  generally  much 
more  cooperative  than  those  who  did  not,  particularly  in  their 
willingness  to  supply  all  the  data  requested.  Those  who  were 
not  offered  gifts  also  withheld  data  more  frequently  than  the 
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average,  though  it  is  not  clear  whether  offer  of  a  gift 
might  have  improved  matters. 

A  pronounced  improvement  in  rapport  was  obtained  on 
the  fourth  wave  of  the  panel  operation  when  a  newspaper 
story  on  the  project  was  sent  to  panel  members  with  the 
advance  letter  and  just  after  all  panel  members  had  been 
sent  a  surprise  gift  of  a  box  of  assorted  cheeses. (4, pp. 26-29) 
While  this  evidence  is  by  no  means  as  conclusive  as  that  from  a  con- 
trolled experiment,  the  fact  remains  that  all  indications  pointed  to  a 
positive  effect  of  compensation  on  cooperation,  as  well  as  on  report  va- 
lidity. On  the  other  hand,  a  very  different  experience  is  reported  by 
Dohrenwend  in  offering  compensation  in  a  small  two-wave  study  in  Manhattan 
seeking  information  on  urban  living  styles. (3)  On  the  first  wave,  half 
the  respondents  were  offered  five  dollars  compensation  while  half  were 
not.  On  the  second  wave,  those  respondents  who  had  received  compensation 
were  offered  an  additional  five  dollars  and  half  of  the  respondents  who 
had  not  received  compensation  on  the  first  wave  were  offered  five  dollars 
for  cooperating  on  the  second  wave.  There  was  no  significant  difference 
in  cooperation  among  the  three  procedures --65  percent  of  households  who 
were  never  compensated  cooperated  on  both  waves  as  compared  to  61  percent 
of  respondents  who  were  offered  compensation  on  both  waves  and  53  percent 
of  respondents  who  were  offered  compensation  for  Wave  2  only.   For  Wave  1 
only,  the  results  were  in  the  opposite  direction,  56  percent  of  respondents 
who  were  offered  compensation  cooperated  compared  to  50  percent  of  those 
who  were  not  compensated.  Since  the  sample  sizes  for  each  of  the  two 
groups  in  the  first  wave  consisted  of  about  80  respondents  and  the  sample 
size  in  Wave  2  was  also  about  80,  none  of  the  differences  is  significant. 
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The  lack  of  effect  of  compensation  may  be  due  to  the  topic  of  the 
study,  which  dealt  with  urban  living  and  attitudes  toward  the  community. 
Unlike  expenditure  or  savings  surveys  that  may  require  substantial  memory 
effort  or  looking  up  records,  most  community  attitude  surveys  are  not  dif- 
ficult or  threatening.  It  may  also  be  due  to  the  manner  of  administration; 
the  compensation  offer  was  made  by  mail  and  referred  to  it  as  an  "honorarium," 
a  word  that  lower  and  middle  income  people  would  find  hard  to  understand. 

c.  Panel  diary  studies 

The  four  studies- reviewed  under  this  heading  were  conducted  by  two 
organizations,  two  by  the  Social  Survey  in  England  and  two  by  the  Survey 
Research  Laboratory  (SRL)  of  the  University  of  Illinois.  The  first  two  • 
studies  related  to  the  United  Kingdom  and  are  reported  by  Kemsley  and 
Nicholson. (5)   In  one  study,  conducted  in  1951  by  the  Social  Survey, 
households  were  asked  to  keep  an  expenditure  record  for  one  week.  The 
sample  was  divided  randomly  into  three  groups  of  about  500  each:   the 
first  group  received  no  compensation,  the  second  group  received  five  shil- 
lings for  cooperating  and  the  third  group  received  ten  shillings.  The 
cooperation  rate  for  contacted  households  was  52  percent  for  those  receiving 
no  compensation,  62  percent  for  those  receiving  five  shillings  and  67  per- 
cent for  those  receiving  ten  shilling 

In  the  second  study,  a  later  and  somewhat  looser  experiment,  respon- 
dents were  offered  25  shillings  to  keep  a  diary  for  four  weeks.  Among  con- 
tacted households,  47  percent  kept  a  diary  for  four  weeks.  Although  there 
was  no  control  group  in  this  survey,  Kemsley  and  Nicholson  speculated  that 
a  higher  cooperation  rate  could  have  been  obtained  had  the  compensation  rate 
been  still  higher,  but  to  our  knowledge  there  have  been  no  additional 
British  experiments  on  compensation  since  their  1960  paper. 
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The  two  experiments  of  SRL  in  obtaining  consumer  expenditures  by 
diary  methods  that  involved  compensation  indicate  that  compensation  in- 
creased cooperation  by  ten  to  15  percentage  points  for  periods  of  two  or 
three  weeks  and  by  about  25  percentage  points  for  four  weeks.  (7,9) 

In  the  first  experiment,  conducted  in  1969  in  the  Springfield  and 
Rock  Island,  Illinois,  Standard  Metropolitan  Statistical  Areas,  respon- 
dents were  randomly  divided  into  four  groups  who  received  the  following 
treatments : 

1.  No  gift.  This  group  did  receive  a  plastic  folder  and  ballpoint 
pen  both  labeled  "University  of  Illinois"  for  use  in  recordkeeping, 
These  items,  which  retailed  for  about  $1.00,  were  not  mentioned 

as  gifts  but  households  were  allowed  to  keep  them. 

2.  Summary  and  comparison  of  purchases.  A  report  of  respondent 
purchases  by  major  categories  and  compared  with  the  purchases  of 
other  panel  families  with  similar  incomes. 

3.  Large  stationery  holder.  A  large  padded  stationery  holder  with 
pen,  retailing  for  about  $5.00,  to  contain  the  diaries  during 
the  recordkeeping  period.  Respondents  were  told  the  holder  was 
a  gift  for  keeping  the  diary. 

4.  American  flag,  posters,  or  government  publications.  A  choice 
of  an  American  flag  with  holder,  posters  of  Illinois  history 
or  one  of  40  popular  Government  Printing  Office  publications, 
again  averaging  about  $5.00  in  value.   (It  is  interesting  that 
roughly  two- thirds  of  this  group  chose  the  American  flag.) 

Within  each  of  these  groups,  respondents  were  asked  to  keep  diaries 
for  periods  of  one,  two,  three,  and  four  weeks.  Thus,  all  respondents 
were  asked  to  keep  a  diary  for  at  least  one  week,  three-quarters  were 


-11- 


asked  to  keep  a  diary  for  at  least  two  weeks,  half  were  asked  to  keep  a 
diary  for  at  least  three  weeks,  and  one-fourth  were  asked  to  keep  a  diary 
for  four  weeks. 

The  percent  cooperating  by  week  and  by  type  of  gift  are  given  in 
Table  1.  Although  the  initial  effect  of  some  gift  on  agreement  to  keep 
a  diary  is  small,  the  differences  in  rate  of  cooperation  are  eight  percen- 
tage points  for  one  week,  13  percentage  points  for  two  or  three  weeks  and 
about  25  percentage  points  for  four  weeks.   Interestingly  enough,  the 
nonmerchandise  offer  (of  providing  purchase  comparisons)  seems  to  have 
been  more  effective  than  the  other  two,  at  least  in  terms  of  cooperation. 

These  results  are  confirmed  by  a  more  recent  1972  study  by  SRL  in 
the  Chicago  SMSA. (9)   In  this  study  respondents  were  asked  to  keep  diaries 
for  two  weeks,  with  a  $5.00  check  offered  to  a  randomized  half.  The  co- 
operation rate  for  households  who  were  compensated  was  79  percent  for 
two  weeks  compared  to  a  cooperation  rate  of  67  percent  for  those  not  com- 
pensated, a  percentage  point  difference  of  12  percent.  This  same  differ- 
ence was  obtained  in  comparing  the  data  for  the  one  week  cooperation  rate-- 
85  percent  for  compensated  households  and  73  percent  for  households  not 
compensated . 

This  most  recent  experiment  provides  some  data  on  the  differential 
effect  of  compensation  by  education  and/or  income  level.   In  this  study, 
which  covered  the  Chicago  SMSA,  significant  differences  were  found  between 
the  effects  of  compensation  in  the  City  of  Chicago  and  in  suburban  areas. 
In  the  suburbs,  compensation  had  no  effect  on  cooperation — 85%  of  contacted 
households  cooperated  regardless  of  cooperation.   In  the  City  of  Chicago, 
85%  of  contacted  households  who  received  compensation  cooperated,  while 
only  68%  of  those  who  received  no  compensation  cooperated. 
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Table  1.  Percent  Cooperating  in  Panel  by  Type  of  Gift 


Extent  of 
Cooperation 

Agreed  to  keep  diary 

Kept  at  least  one 
diary 


No 

gift 


Type  of  Gift 


Summary  and   Large 
Some     comparison    stationery   Flag  or 
gift     of  purchases  holder      book 


85.7(113)   89.6(412)   88.3(128) 


77.1(113)   84.7(412)   85.9(128) 


92.0(125)   88.7(159) 


90.4(125)   79.2(159) 


Kept  at  least  two 
diaries 


62.1(90)   75.1(297)   71.1(90) 


82.6(101)   71.3(106) 


Kept  at  least  three 
diaries 

Kept  four  diaries 


54.1(61)   67.0(198)   66.7(63) 
23.1(30)   48.6(96)    54.8(31) 


70.2(66)   64.3(69) 
39.4(31)   51.4(34) 


Note:  Percentages  in  this  table  are  the  number  of  households  keeping  given 
number  of  diaries  as  a  percent  of  those  asked  to  keep  a  diary  for  at  least  that 
length  of  time,  for  that  type  of  gift.  The  base  figures,  in  parentheses,  are 
the  number  of  households  asked  to  keep  a  diary  for  at  least  that  length  of  time, 
Thus,  the  90  keeping  two  diaries  and  who  received  no  gift  is  the  number  of  house- 
holds asked  to  keep  the  diary  for  2,  3,  or  4  weeks. 

Source:   (7, p. 728) 


-13- 


The  obvious  differences  between  city  and  suburban  households  are  in 
the  two  social  class  variables,  income  and  education.  Lower  income  city 
households  generally  find  record-keeping  more  difficult  while  at  the  same 
time  the  compensation  they  get  has  greater  marginal  utility.  Thus,  com- 
pensation helps  to  reduce  the  panel  biases  against  low  income  households. 
Panels  generally  get  lower  cooperation  from  very  small  households  (1  or 
2  members)  and  very  high  income  households,  but  compensation  seems  to 
have  no  effect  on  these  biases. 

Since  low  income  households  spend  less,  the  mean  expenditures  with 
these  households  included  will  be  lower  than  if  they  are  not  in  the  panel. 
These  results  are  confunded,  however,  by  the  results  of  the  next  section 
that  indicate  that  compensation  may  influence  the  accuracy  of  reporting. 

Auspices.  What  limited  data  are  available  indicate  that  compensation 
has  the  same  effects  of  increasing  cooperation  regardless  of  auspices. 
In  the  1972  study  in  the  Chicago  SMSA  just  discussed, (9)   for  half  the 
sample,  the  advance  letter  was  on  U.S.  Bureau  of  the  Census  stationery  and 
signed  by  its  Director.  The  diaries  also  had  the  Bureau  of  the  Census 
headings  on  them,  and  interviewers  said  that  they  were  acting  as  collecting 
agents  for  the  Bureau  of  the  Census.   In  the  other  half  of  the  sample, 
advance  letters,  diaries,  and  interviewers  were  all  identified  as  from 
the  University  of  Illinois.   In  the  City  of  Chicago,  for  University  of 
Illinois  auspices,  compensation  increased  the  cooperation  rate  14  percent- 
age points  from  75%  to  89%,  and  for  Census  Bureau  auspices,  compensation 
increased  cooperation  17  percentage  points,  from  64%  to  81%.  Compensation 
had  no  effect  in  the  suburbs  in  either  case. 

All  things  considered,  the  evidence  from  these  studies  would  seem 
to  support  the  hypothesized  effects  of  compensation,  at  least  as  applied 
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to  cooperation.  The  evidence,  though  relatively  sparse,  strongly  supports 
the  positive  effects  of  compensation  on  cooperation  in  the  case  of  diary 
studies  with  the  principal  effects  being  in  areas  more  heavily  populated 
with  lower  income  and  more  poorly  educated  households.  The  evidence  is 
more  mixed  in  the  case  of  personal  interview  studies. 

4.  Effect  on  Validity 

Little  information  is  available  on  the  effect  of  compensation  on 
report  validity,  partly  no  doubt  because  of  the  difficulty  of  obtaining 
the  necessary  validating  data.  As  a  result,  perhaps,  some  of  the  past 
studies  have  not  considered  this  question  at  all.  Yet  in  many  ways,  this 
is  the  key  question,  for  one  may  argue  that  the  ultimate  objective  of  com- 
pensation is  not  to  raise  response  rates  for  their  own  sake,  but  as  a  means 
of  improving  the  validity  of  the  resulting  information. 

In  theory,  there  need  be  no  relation  between  the  cooperation  effect 
and  the  validity  effect.  Thus,  if  compensation  brings  about  no  increase 
in  response  rate  but  a  substantial  improvement  in  the  validity  of  the  data 
reported  by  those  who  do  cooperate,  the  extra  cost  and  effort  could  be 
judged  well  worth  it;  whereas  if  the  response  rate  were  to  rise  markedly 
but  with  no  improvement  in  data  quality,  the  use  of  compensation  would 
be  highly  questionable.   It  is  therefore  all  the  more  unfortunate  that  so 
few  results  are  available  on  this  aspect,  and  many  of  these  are  only  cir- 
cumstantial. 

A  priori  one  would  expect  compensation  to  improve  report  validity, 
in  the  sense  that  the  offer  of  compensation  conveys  to  the  respondent  a 
moral  obligation  to  devote  more  effort  or  thought  to  providing  the  re- 
quested information,  whether  from  memory  or  by  diary  methods.  However, 
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on  this  basis  it  could  be  argued  that  the  effect  should  be  greater  where 
diary  methods  are  used  than  recall  by  personal  interview,  because  greater 
effort  is  required  to  keep  diaries,  except  if  the  respondent  is  being 
asked  to  furnish  data  from  records. 

With  these  thoughts  as  a  frame  of  reference,  let  us  look  at  what 
the  data  have  to  show.  As  will  be  seen,  it  is  particularly  difficult 
to  draw  any  firm  conclusions  in  view  of  the  very  few  studies  and  the  lack 
of  validating  information  on  expenditures.  The  absence  of  the  latter  is 
expecially  frustrating,  for  it  means  that  data  validity  has  to  be  inferred 
indirectly.  The  criterion  used  in  such  cases  is  that  "more  means  better." 
In  other  words,  more  reports  of  expenditure  and  higher  outlays  per  house- 
hold are  considered  to  reflect  more  valid  data,  on  the  premise  that  ex- 
penditures are  being  reported  that  would  otherwise  be  omitted.  This  has 
some  basis  in  the  well-known  tendency  of  household  survey  data  to  under- 
state expenditure  aggregates  but  does  not  make  any  allowance  for  tele- 
scoping effects  that  can  act  in  the  opposite  direction.   Indications  from 
our  data,  however,  are  that  the  former  effect  is  predominant. (8) 

Four  of  the  studies  reported  earlier  contained  some  information  re- 
lating to  validity  effects,  and  those  aspects  of  those  studies  are  reviewed 
here.   In  the  one-time  personal  interview  Survey  Research  Center  study 
reported  by  Lansing,  Ginsburg  and  Braaten(6),  no  significant  differences 
in  reporting  of  savings  accounts  were  found  between  compensated  and  non- 
compensated households,  because  of  small  sample  sizes.  Nevertheless,  the 
results  reproduced  from  that  study  in  Table  2  indicate  a  slightly  higher 
tendency  for  heads  of  households  who  were  compensated  to  report  ownership 
of  a  savings  account,  and  to  report  more  accurately  the  balance,  in  the 
account,  if  they  were  willing  to  state  a  balance,  than  non- compensated 
heads. 
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The  personal  interview  savings  panel (4)  described  on  pp.  7-8  yielded 
circumstantial  evidence  of  improved  data  accompanying  the  offer  of  com- 
pensation. Despite  the  lack  of  controls,  the  evidence  was  fairly  strong 
that  among  lower  income  households,  offer  of  compensation  increased  their 
willingness  to  report  on  their  saving  behavior  both  initially  as  well  as 
at  a  later  stage  in  the  panel.  The  findings  of  that  study  suggest  that 
the  impact  of  compensation  wears  off  over  time  and  that  a  follow-up  gift 
is  well  worth  considering. 

The  two  SRL  diary  experiments  in  obtaining  expenditure  data  yielded 
mixed  results  on  the  effect  of  compensation  on  report  validity.  (7,9) 
In  the  1969  SRL  study,  households  who  received  some  gift  reported  higher 
levels  of  expenditures  than  those  who  received  no  gift,  as  shown  in  Table 
3.  The  differences  increased  each  week,  from  nine  percent  the  first  week 
to  18  percent  the  second  week  to  about  60  percent  in  weeks  three  and  four. 
Over  all  four  weeks,  households  receiving  a  gift  reported  17  percent  more 
expenditures  than  did  households  who  did  not  get  a  gift. 

These  results  were  not  confirmed  in  the  1972  SRL  study. (9)  Control- 
ling for  city-suburban  and  diary-phone  procedures,  no  significant  differ- 
ences were  found  in  total  expenditures,  for  food  products  only,  or  by 
individual  product  class  types. 

We  can  only  speculate  why  compensation  affected  reporting  on  the 
earlier  experiment  but  not  on  the  later  one.  The  earlier  experiment 
tested  periods  up  to  four  weeks,  and  the  greatest  differences  between 
compensated  and  non- compensated  households  were  found  in  the  third  and 
fourth  weeks.  Thus,  shortening  the  recordkeeping  period  reduces  the  ef- 
fects of  compensation  on  level  of  diary  reporting.  Still  for  the  first 
two  weeks  combined,  there  was  more  than  a  ten  percent  difference  in 
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Table  3;  Reported  Total  Expenditures  Per  Household 
By  Type  of  Gift  and  Week* 


Week 

No  Gift 

Some 

Gift 

Summary  and 
comparison 
of  purchases 

Large 
stationery 
holder 

Flag 

or 
book 

1 

$124 

C85) 

$135 

(327) 

$141 

C97) 

$136 

C105) 

$129 

(125) 

2 

107 

(42) 

126 

(207) 

122 

(57) 

113 

(74) 

142 

(76) 

3 

89 

(20) 

142 

(120) 

144 

(37) 

140 

C41) 

141 

(42) 

4 

61 

C7) 

96 

(46) 

96 

(13) 

76 

(16) 

115 

(17) 

Four  week 
average  $112 

C154) 

$131 

(700) 

$133 

(204) 

$126 

(236) 

$133 

(260) 

Source:  (7, p. 732) 
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level  of  reported  expenditures  between  compensated  and  non- compensated 
households  on  the  earlier  experiment,  as  compared  to  no  difference  on  the 
later  experiment.  Aside  from  sampling  errors,  there  may  have  been  some 
interviewer  effects  since  the  previous  study  stressed  gifts  as  a  major 
variable,  with  three  different  gifts  being  tested,  while  in  the  1972  study 
the  major  variable  was  the  use  of  telephones. 

5.  Nature  of  Compensation 

Form.  There  is  little  evidence  to  indicate  that  the  form  of  compen- 
sation has  any  significant  effect  on  cooperation  or  accuracy  of  reporting. 
A  wide  variety  of  gifts  as  well  as  money  have  been  used  for  compensation, 
with  no  strong  evidence  of  differential  effects.  This  was  illustrated  in 
Table  2  by  the  response  rates  for  three  different  forms  of  compensation 
given  in  the  1969  SRL  diary  experiment. (7)  While  the  large  stationery 
holder  obtained  slightly  higher  cooperation  rates  than  the  other  gifts 
in  the  first  three  weeks,  none  of  the  differences  were  significant.   In 
the  fourth  week,  these  cooperation  rates  reversed  and  cooperation  was 
lowest  for  households  who  received  the  stationery  holder. 

Other  non-experimental  evidence  also  indicates  no  differences  in 
cooperation  by  type  of  compensation.  The  1972  SRL  experiment  used  money 
for  compensation,  as  did  the  British  Survey  reported  by  Kemsley  and 
Nicholson  (5) .  There  were  no  substantial  differences  in  cooperation 
between  these  studies  and  the  results  in  Table  1.  Also,  MRCA  uses  gifts 
while  Nielsen  uses  money  and  both  panels  have  about  the  same  level  of 
cooperation. 

The  results  of  Table  3  indicate  that  the  form  of  the  gift  has  no 
effect  on  the  level  of  reporting.   For  the  four  weeks,  the  average  weekly 
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expenditure  did  not  vary  by  more  than  an  insignificant  five  dollars  from 
the  mean  by  type  of  gift.  Obviously,  it  would  be  possible  to  bias  expen- 
diture data  by  gifts  that  influenced  future  purchase  behavior.  For  this 
reason,  gifts  are  never  chosen  from  the  same  expenditure  categories  as 
are  being  measured.  Thus,  if  one  were  measuring  food  consumption,  one 
would  avoid  gifts  of  food  or  related  to  food  preparation. 

Level  and  Frequency.   If  recordkeeping  is  considered  onerous,  one 
would  expect  that  cooperation  would  increase  with  increased  compensation. 
The  relation  is  probably  curvilinear,  however.  Respondents  have  a  vague 
idea  what  their  efforts  are  worth,  and  payments  far  above  this  level 
might  well  result  in  suspicion  by  respondents  and  anxiety  among  inter- 
viewers that  would  reduce  cooperation.  Similarly,  compensation  much 
below  the  expected  level  might  be  treated  as  no  compensation.  In  the 
range  of  reasonable  compensation  levels,  the  cooperation  rate  may  be 
fairly  flat. 

Tiie  only  published  evidence  we  know  of  is  the  Kerns  ley  and  Nicholson 
study (5)  discussed  earlier  which  indicated  that  response  increased  from 
62  to  67  percent  as  the  compensation  level  increased  from  five  to  ten 
shillings.  When  the  time  period  was  increased  to  four  weeks,  however, 
increasing  the  compensation  to  25  shillings  still  results  in  a  lower  co- 
operation rate. 

Nothing  is  published  about  the  effects  of  level  of  compensation  on 
accuracy  of  recording  expenditures.  One  suspects,  however,  that  accuracy 
is  less  sensitive  to  level  of  compensation  than  is  cooperation,  so  that 
changes  in  compensation  that  do  not  affect  cooperation  would  not  affect 
recording  levels. 


-21- 


Perhaps  more  important  than  level  of  compensation  is  the  frequency 
of  reinforcement  in  the  case  of  panel  studies  covering  periods  longer 
than  one  or  two  weeks.  MRCA  and  Nielsen  reward  respondents  each  time  a 
diary  or  cartridge  is  mailed.  The  SRL  results  in  Tables  1  and  3  suggest 
that  an  additional  gift  in  the  fourth  week  might  have  increased  cooper- 
ation. The  results  of  the  1957  savings  panel  experiment,  discussed  on 
pp. 29-30,  suggest  strongly  that  follow-up  gifts  can  have  noticeable  ef- 
fects on  respondent's  cooperativeness,  especially  in  a  panel  study  ex- 
tending over  many  months. 

6.  Summary  and  Implications 

As  must  be  evident  from  this  sketchy  review,  the  effects  of  compen- 
sation on  consumer  expenditure  or  saving  data  have  been  the  subject  of 
relatively  few  studies.  Since  this  past  work  has  been  mostly  in  the 
nature  of  case  studies,  any  results  cannot  be  considered  definitive  but 
rather  more  in  the  nature  of  hypotheses  for  further  investigation. 

What  does  seem  to  emerge  is  that  the  effects  of  compensation  in 
such  studies  vary  depending  on  whether  the  study  is  based  on  one-time 
interviews  or  on  a  panel  approach,  and  depending  on  whether  or  not  the 
sample  members  are  asked  to  keep  written  records.   For  one-time  interviews 
with  no  written  records,  there  is  no  evidence  that  compensation  improves 
the  results  either  from  the  point  of  view  of  cooperation  or  report  validity. 
It  should  be  stressed,  however,  that  this  inference  is  based  on  only  two 
studies  one  of  which  does  not  seem  to  have  sought  detailed  expenditure 
information. 

In  panel  studies,  whether  by  personal  interview  or  by  written  record- 
keeping, there  are  indications  that  some  form  of  compensation  will  con- 
tribute to  a  higher  rate  of  response  as  well  as  to  more  complete  and 
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accurate  information.  To  judge  from  the  experiences  reported  from  the 
one  personal  interview  panel  covered  in  this  paper,  rapport  with  panel 
members  can  be  improved  greatly  by  offering  some  form  of  compensation 
after  three  or  four  interviews.  Continuing  interaction  with  the  panel  mem- 
bers either  in  the  form  of  compensation  or  by  offering  them  reports  on 
earlier  results  or  other  types  of  information  seems  to  improve  willing- 
ness to  participate  in  the  panel. 

The  effects  of  compensation  are  especially  clear  in  obtaining  higher 
response  for  even  short  periods  of  time  if  sample  members  are  requested 
to  keep  written  records  of  expenditures.   In  all  three  studies  covered 
in  this  paper,  marked  increases  in  the  rate  of  response  seem  to  have 
occurred  as  a  result  of  compensation.  These  increases  were  concentrated 
mostly  among  families  with  lower  incomes  and  less  education.  Despite 
this  clear  effect  on  the  rate  of  response,  however,  the  apparent  effects 
on  report  validity  are  mixed.  Whereas  in  soaie  instances  compensation  ap- 
parently induced  higher  report  validity,  in  other  instances  it  seemed 
to  have  no  effect.  The  best  that  can  be  said  in  the  latter  case  is  that 
in  such  instances  the  initial  cost  of  compensation  seems  to  be  offset  by 
reduced  field  costs  brought  about  by  the  higher  rate  of  response,  so  that 
neither  total  cost  nor  cost  per  unit  of  information  is  affected  adversely 
by  compensation. 

One  finding  that  runs  counter  to  the  hypotheses  advanced  on  page 
six  is  that  form  of  compensation  seems  to  have  little  effect  on  either 
the  rate  of  response  or  report  validity.  To  be  sure,  no  clearly  commer- 
cial offerings  (such  as  a  subscription  to  a  popular  magazine)  were  made 
in  any  of  these  studies,  but  it  does  appear  that  for  a  wide  range  of  re- 
latively non-commercial  possibilities  the  effect  of  compensation  is  not 
likely  to  be  influenced  by  the  specific  form  used. 
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In  view  of  these  findings  and  inferences,  one  may  well  ask  from  a 
practical  survey  point  of  view  when  and  under  what  conditions  compensation 
would  seem  advisable  in  studies  of  this  type.  In  our  opinion,  there  does 
not  seem  to  be  much  basis  at  the  present  time  for  offering  compensation 
on  one-time  interviews.  There  does,  however,  seem  to  be  a  clear  rationale 
for  offering  compensation  on  a  panel  study,  whether  or  not  information 
is  sought  by  written  recordkeeping  by  the  respondents,  and  whether  or  not 
the  study  is  being  conducted  by  a  governmental  or  non-profit  agency.  In 
terms  of  cooperation  the  results  are  likely  to  be  much  better,  while  in 
terms  of  report  validity  the  results  are  likely  to  be  at  least  as  good 
as  if  no  compensation  were  offered.  The  cost  analyses  from  the  two  SRL 
diary  studies  indicate  that  the  higher  rates  of  response  brought  about 
by  the  compensation  serves  to  more  than  offset  their  costs,  with  the  result 
that  the  total  cost  of  the  survey  and  the  cost  per  interview  is  not  in- 
creased, and  may  even  decrease.  Thus,  the  interviewer  and  field  supervisory 
costs  to  recruit  a  household  to  keep  a  diary  for  two  weeks  are  about  $30 
at  current  rates  of  $2.50  per  hour  for  interviewers.  To  recruit  fifteen 
additional  households  per  hundred  would  cost  about  $450.  The  compensation 
costs  for  all  75-80  recruited  households  per  hundred  would  cost  $400  or 
less  at  a  rate  of  $5  per  household. 

As  mentioned  at  the  beginning  of  this  section,  these  results  are 
based  on  sparse  evidence  and  much  more  research  is  needed  on  this  question. 
In  particular,  very  little  information  exists  on  the  effects  of  varying 
the  amount  or  the  frequency  of  compensation  on  the  two  criteria  we  have 
used  in  this  paper,  as  well  as  on  cost.  Research  is  also  needed  on  the 
effect  of  the  time  period  on  these  results  as  well  as  of  the  type  of 
approach  and  the  methods  of  interviewer  training. 
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Finally,  no  work  whatsoever  seems  to  have  been  done  to  investigate 
the  effects  of  compensation  in  rural  areas.  We  suspect  that  such  effects 
will  also  depend  on  the  educational  and  income  levels  of  the  population, 
and  will  hence  be  greater  in  the  lower  income  areas,  but  on  the  other 
hand  these  effects  may  be  mitigated  by  the  traditionally  better  cooper- 
ation obtained  on  almost  all  types  of  surveys  in  rural  areas. 
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